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Abstract 

This paper is a study of continuous time 
Singular Spectrum Analysis (SSA). We 
show that the principal eigenfunctions 
are solutions to a set of linear ODEs 
with constant coefficients. We also in- 
troduce a natural generalization of SSA, 
constructed using local (Lie-) transfor- 
mation groups. The time translations 
used in standard SSA is a special case. 
The eigenfunctions then satisfy a sim- 
ple type of linear ODE with time de- 
pendent coefficient, determined by the 
infinitesimal generator of the transfor- 
mation group. Finally, more general one 
parameter mappings are considered. 

Singular Spectrum Analysis (SSA) is a rela- 
tively recent method for nonlinear time series 
analysis. The original idea behind SSA was 
first presented by Broomhead and King |IJ] , in 
the context of time series embedding. Dur- 
ing the last decade this technique has been 
very successful and has become a standard tool 
in many different scientific fields, such as cli- 
matic [||, meteorological j|, and astronomi- 
cal [[| time series analysis. For introductions 
to the SSA technique, see e.g., [|| |J. 

In practical applications, a time series is a 
result of a sampled measurement, and is there- 



fore discrete. This paper is a theoretical study 
of SSA, and it is therefore more natural to con- 
sider the general case of continuous time. We 
start the paper by a (semi) formal expansion 
of the SSA procedure to continuous time. 

Let f(t) be a function representing a con- 
tinuous time signal on an interval £It = [0, T]. 
We assume that f(t) € £ 2 (^t), where £ 2 (^t) 
is a Hilbert space with an inner product de- 
fined as (f,g)n T — J Qt dtf(t)g(t) and a norm 

ll/lln = y/(f, f)n- We define a trajectory func- 
tion x(£,t) 

X{Z,t) = /(£ + «) (i) 

where t € Vt w = [0,W] and £ S 
n T -w = [0,T-W]. The parameter W is 
fixed and referred to as the window length. 
By construction X(£,t) € /^(^t-w) x 
C^i^w) and the norm is defined as ||A^|| 2 = 
L T _ w dUn w dtX(U)mT). 

A Schmidt decomposition (the continuous 
equivalent to a Singular Value Decomposition 
of a matrix) of the trajectory function is defied 
as 

d 

x&t) = SVW£W(*) (2) 

k=l 

where d is referred to as the rank (which is 
often infinite). All partial sums, k = 1 to d, are 
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optimal d-rank /^-approximations of X(£, t), 
i.e., the functions t>fe(£) and Uk(t) fulfills 

d 

min||X(£,i)- VTWeWWII (3) 

u.v ^ — » 

fe=l 

Without lost of generality, the functions Vfc(£) 
and Uk(t) can be assumed to be normalized, 
\\uk\\ = \\vk\\ = 1- Eq. | implies 

v k (0 = [ dtX(^t)^klt) 

V M Jflw 

Uk(t) = [ d£X{te)wffi (4) 

V A*.. JriT-w 

together with the orthogonality conditions 
(vk,vi) QT _ w = S k i and (u k ,Ui) QT = S k i- The 
two relations in Eq. ^ can be combined into 
eigenvalue problems for the functions Uk (t) re- 
spective ffc(£) 

I dt'S{t,t')u k {t') = X k u k (t) 

I dZ'R{M)v k {Z') = X k v k (0 (5) 

where the t-covariance function S(t,t') and 
the ^-covariance function R{£,£') are defined 
as 

R(Z,0 = f dtx(u)xW~t) 

Ja. w 

S(t,t') = f dix{i,t)Y(^¥) 

From Eq. || it is clear that and 
Uk(t) are eigenfunctions to two compact, lin- 
ear and symmetric integral operators R and 
S with kernels and S(t,t'). The 

spectral theorem then guarantees and 
Uk(t) to form complete orthogonal bases 
in £ 2 (n T _ w )/Af(R) respective C 2 (n T )/M(S) 
(where A/"(-) denotes a null-space). The eigen- 
values Afc are real and non-negative, and mea- 
sure the variance of X(£,i) in the "direction" 



defined by ffc(£) and Uk{t). Note the close cor- 
respondence to principal component analysis. 

By construction, SSA decomposes the orig- 
inal time series into orthogonal components. 
Usually, the components also represent intu- 
itive contributions to the time series, such as 
a trend, various oscillation modes and noise. 

We start our analysis of the SSA technique 
by investigating signal separation into orthog- 
onal subspaces. From now on, due to space 
limitations, we will focus our attention to 
the "right eigenmodes" , Uk (t) . By symmetry, 
equivalent results are valid for the "left eigen- 
modes", Vk(£,)- Assume that the time series 
can be decomposed as f(t) = J^i where 

/ de/i(* +0^(^+0 = cw« 

( dtfi(t + + = C 2 -S t3 (6) 

Vt,t' € ttw,€,C e n T _ w . Then the t- 
covariance kernel also decompose as S = 
where Si corresponds to fi. Since 
Jn w dtS dW)Sj{t',t") = if i ^ j, it fol- 
lows that SiU — Xu implies u £ J\f(Sj), 
j =/= i, X =^ 0. An eigenfunction of Si is there- 
fore also an eigenfunction of S, i.e., SiU = Xu 
implies Su — Xu. Since S a symmetric opera- 
tor, Ui _L Uj when ^ Xj, which guarantees 
that a Schmidt decomposition is unique up to 
a rotation of eigenfunctions with identical sin- 
gular value. It follows that if f(t) = J2i /*(*)) 
Eq. H is fulfilled and fi(t) and fj(t) have dis- 
joint spectra, then the SSA decomposition in 
Eq. |^ is a direct sum of the decompositions of 
the individual time series. In this case fi(t) 
and fj(t) are called strongly separable. If the 
spectra are not disjoint, but Eq. |^ is fulfilled, 
fi(t) and fj(t) are called weakly separable (us- 
ing the same notation as in |(|). 

An important implication of the above anal- 
ysis is that, for periodic functions, when the 
total time frame T and the time window W is 
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chosen such that T — W — nr and W = n'r 
(where r is the period and n, n' € Z + ), the sin- 
gular spectrum yf\k is identical to the Fourier 
coefficients of f(t). Furthermore, if the Fourier 
coefficients are distinct, then the eigenfunc- 
tions Mfc (t) are identical to the basis functions 
in the Fourier expansion. Similar results hold 
asymptotically in the infinite time frame limit. 

To understand the SSA procedure, it is es- 
sential to further analyze the characteristics of 
the orthogonal eigenfunctions Uk(t). We start 
by the following trivial observation: 

/(« + £') + *) = /« + (£' + *)). (7) 
Using this relation in Eq. || gives 

d 

fc=i 

d 

fe=i 

Apply the projection operator J Q d£vt(£), 
use the orthogonality of Vfe(£), subtract ui(t), 
divide by and finally let £' — > 0. Tech- 
nically, a problem appears when integrating 
v k{€ + £') over £ G ^t-iv- The function is not 
defined when £ + £' ^ flr_w. However, since 
£' — > 0, using any smooth continuation of Vk 
gives equivalent results. Alternatively, Eq. |^ 
could be replaced by d t /(£ + 1) = 9g/(£ + f). 
Either way, we find: 

d 

d t u k (t) = 5^A w u,(t), (8) 

where 

A fei = / de« fc (£)0e«i(0 

V Afc JQ T _ W 

does not depend on i. Note the connection 
between A and the cross-correlation function. 



Projecting Eq. || onto also gives: 

Am = / dtui(t)d t u k (t) 
Jn w 

which clearly shows the symmetric relation be- 
tween the left and right principal eigenfunc- 
tions. 

From Eq. || it is clear that polynomial, ex- 
ponential and harmonic functions show espe- 
cially simple (finite) spectra during SSA. In 
the more general case, the SSA procedure de- 
composes the time series into an optimal (in- 
finite) linear combination of polynomial, ex- 
ponential and harmonic functions. This has 
previously been discovered by others, but the 
approach taken in this paper is quite differ- 
ent and arguably more straight forward. For 
a discussion on related work see ||, || . 

We now show a connection between the ma- 
trix A and an underlying dynamical system 
from which the time series is generated. Let 
f(t) be a solution to a linear system of differ- 
ential equations 

d t f(t) = Mf(t), 

and let a scalar time series f(t) be defined by 
some linear projection of f (t), f = p T f, where 
p is non-degenerate in the sense that all os- 
cillation modes that appears in f(t) are also 
present in f(t). We shall now prove that M 
and A have the same spectra. The vector func- 
tion f(t) can be decomposed as: 

d , — 

= ^v**™* 5 *®' ( 9 ) 

fe=l 

where t is restricted to the interval ttw- Since 
f(£ + t) = exp(Mf)f(t), we have 

d , 

/(£ + *) = E v A * (p T ex p w «*(*)• 
fc=i 

Comparing this expression to Eq. |^ and use 
the uniqueness of the SVD expansion, gives 
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vk(0 = C k ■ Y.i R ki (p T exp {M£} w{), where 
Cfc is a normalization constant and R is some 
rotation matrix which rotates elements within 
the equivalence classes defined by identical sin- 
gular values. This further shows that u(t) = 
Ruff), and therefore the time derivative of 
Eq. |] gives: 



y X k Mw k R M ui(t) = 

kl 

^ \f^kWkRklAlmU m (t)- 

kirn 

Using the orthogonality of uu (t) and , gives 

A = R T A- 1 W T MWAR, (10) 

where we use the matrix notation: A = 
diag ^\/Ai, \/a2, • • , W = (wi,w 2 , ...)• 

Since (\VAr) (r t A- 1 W t ) = 1, Eq. ^ im- 
plies that A and M have identical spectra. 

Note that this line of argument is similar to 
the local linear analysis of dynamical systems, 
used as part of the proof delay coordinate em- 
bedding theorems J7J . It also shows a straight 
forward connection between PCA of a set of 
time series from a system of ODEs, and a SSA 
analysis of a projection from the system. 

Though natural in the discrete case, the def- 
inition of the trajectory function as A"(£,i) = 
/(£ + is somewhat arbitrary in our analy- 
sis. Why not X(£,t) — f(£t), for example? 
We can use this arbitrariness both to gener- 
alize SSA and to gain better theoretical un- 
derstanding of the procedure. Recall the def- 
inition of a (local) transformation group (see 
e.g., §: 

Definition 1 A transformation group is a 
continuous Lie group G and a set M C R™ 
along with a smooth map $ : Gx M -» M 
which satisfies, for every g, h £ G, x € M , 



together with the existence of an identity ele- 
ment e and an inverse g^ 1 for all g G G. 

In this paper, the elements of the transforma- 
tion group is spanned by a parameter £ (we 
have a one-parameter group). We will some- 
times use the compact notation: 

We also chose the parametrization such that 
3(0) = e. 

A one-parameter transformation group de- 
fines a vector field V = Yli d( x i)d Xi '- 
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which formally is "solved" as 

y(£,x) = exp{£V}x. 

The vector field is therefore called the in- 
finitesimal generator of the one-parameter 
transformation group. Note that the exponen- 
tial map also implies <?(£) o g(£') = <?(£ + £'). 

The transformation group also generates or- 
bits through every point x £ M, defined as 
(/>(£) = \I/(£, x). The orbits are solutions to a 
system of ordinary differential equations: 



cm)) 



(12) 



where the explicit representation of the vector 
field is used. Eq. [l2] can be used to derive ex- 
plicit expressions for the transformation group 
corresponding to a vector field. 

Using this framework, the SSA procedure 
may be generalized in the following way. Con- 
sider a trajectory function Xq, constructed 
from a one-parameter transformation group 
\&j acting on M = £lw, with the continuous 
Lie group G: 



V(goh,x), (11) 



(13) 
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where ^ : Qt-w X — * ^t- Eg pi] provides 
a relation equivalent to Eq. [?]: 

Eq. |lj can be used to find a system of differ- 
ential equations, equivalent to Eq. |8|, in terms 
of the infinitesimal generator of the transfor- 
mation group: 



(15) 



1=1 



where C [■] expresses a Lie derivative with re- 
spect to the transformation group. Since Uk(t) 
is a scalar function (the index k is fixed), 
£ [ u fc] = C(t)dtUk(t), where £(*)c?« is the vec- 
tor field generating the one-parameter trans- 
formation group. The matrix A is now defined 
as 



A 




Again, A is independent of t. If the time series 
is defined as a linear projection / = p T f and 
C [f (t)] = Mf, an equivalent analysis as above 
shows that M and A have identical spectra. 

Lie groups was originally developed to an- 
alyze symmetries in differential equations, see 
e.g., ||. To make a direct connection between 
this theory and the analysis above, assume 
that f(t) is a solution to some linear differ- 
ential equation, i.e., D t f(t) — where D t is 
a linear operator. Assume further that is 
a symmetry group of the differential operator 
D t , then D t f(t) = implies D t (# € /(f)) = 0. 
If the transformation group is used to con- 
struct the trajectory function, then the prin- 
cipal eigenfunctions Uk(t) will also satisfy the 
same differential equation, D t Uk{t) = 0. This 
can be seen by noting that the t-covariance 
kernel can be written explicitly as 



S(t,t') 



dU (* f t) /(* C f) • (I 6 ) 



Since D t S (t, t') = 0, it follows from the eigen- 
value problem in Eq. || that D t Uk{t) = 0, if 
A fe ^ 0. 

The analysis also shows how the global 
transformation group, used in the construction 
of the trajectory function, affects the eigen- 
modes via the infinitesimal generator. Let the 
vector field be defined by V = C,{t)d t . The re- 
lation between the eigenmodes in Eq. 15 and 
Eq. H is then given by a variable transforma- 
tion, r 
the form 



— J ^fy. The general solutions is on 



u(/-) = <<xi> <J A / \ u„. (17) 



Eq. 17 reflects the fact that up to an iso- 
morphism, there are only two connected one- 
parameter Lie groups, R and 50(2), corre- 
sponding to real respective imaginary eigen- 
values of A, see || for details. 

In the standard SSA, time translations are 
used to construct the trajectory function, cor- 
responding to ^(^,t) = £ + t, V = dt and 
u(i) = cxp{At}uo. Using scale transforma- 
tions, ty(£,t) — e^t, corresponds to V = tdt 
and u(i) = exp {A ln(t)} Uo, which contain 
functions on the form t a (scaling functions) 
and sin (oj ln(i)). 

In fact, any smooth vector field generates 
a transformation group, which may only be 
locally defined. For example, consider C,(t) = 
t a for some constant a € R, a ^ 1. Using 
Eq. |l| and Eq. [H], we then have: 

V = t a d t 

= + 

u(i) = axp{A/? _ V}u , 

where = 1 — a. 

For completeness, we finish this paper by a 
further generalization of the SSA. Consider a 
smooth function tp : Qw X Qt-w ~ > ^t, and 
construct the trajectory function as X(£, t) = 
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f(if>(£,t)). We assume the mapping to be of 
maximal rank, i.e., have a non- vanishing Jaco- 
bian. The implicit function theorem then en- 
sures that in the neighborhood of each point 
on a curve defined by ip(£,t) = const., £ can 
be expressed as a function of t and vice versa, 
i.e., the mapping ip defines an implicit one- 
dimcnsional submanifold in R 2 . We assume 
tjj(0,t) — t. For small we then use the 
smoothness of ip to write 

V> (€',*)) = iH£ + M*)s(£,*)(OV) 

^,t) = t + fe(f)(£T, (18) 

which is valid to order k > 1 in £' for some 
functions h and g. The derivative of the first 
relation in Eq. |l8| gives an explicit expression 
for 



Note that if h(t)g(£,t) = 1 and k = 1, then 
^ is a transformation group according to Def- 
inition 0. Given a function g(^,t) we can also 
find the global mapping if) by solving the linear 
first order PDE (using for example the method 
of characteristics): 

rl>(0,t) = t, (19) 

corresponding to Eq. |lj above. Using Eq. |lj 
we derive the structural differential equations 
for the principal eigenfunctions 



d t u k (t) 



£A w (i)u,(t) (20) 



equations hard to analyze in general. Again 
it worth noting that an equivalent approach 
would be to use the relation 

d&it, (^(e, *)) = d t m t)dj (^(e, *)) , 

which is valid for all smooth functions / (es- 
pecially f(x) — x which gives Eq. pjj). 

There are a number of different ways to 
further generalize SSA along the lines of this 
paper. The perhaps most natural is to con- 
sider multi-dimensional transformation groups 
and/or multi-dimensional data fields. This 
would require an extension of the SVD to ten- 
sor decomposition. This is not straightfor- 
ward P|, but generalized versions of the SVD 
does exist pp[ . 
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